Addendum: out of curiosity, I ran some comparisons between the built-in SQLite and the SQLCipher build with the following script (and adding a GRDB_PERFORMANCE_TESTS check to enable GRDBPerformanceTests in Package.swift).

The results over a dozen runs are a fairly consistent slowdown of around 10% when using SQLCipher. It'd be interesting to tweak the SQLite build flags in swift-sqlcipher to see what might increase/decrease the performance.

echo "With GRDBCIPHER"; rm -r .build; GRDB_PERFORMANCE_TESTS=1 GRDBCIPHER="https://github.com/skiptools/swift-sqlcipher.git#1.2.1" swift test --filter GRDBPerformanceTests 2>&1 | grep --after-context=2 "Test Suite 'GRDBPackageTests.xctest' passed"
echo "Without GRDBCIPHER"; rm -r .build; GRDB_PERFORMANCE_TESTS=1 swift test --filter GRDBPerformanceTests 2>&1 | grep --after-context=2 "Test Suite 'GRDBPackageTests.xctest' passed"

With GRDBCIPHER
Test Suite 'GRDBPackageTests.xctest' passed at 2025-01-26 22:33:13.477.
	 Executed 23 tests, with 0 failures (0 unexpected) in 353.144 (353.146) seconds
Test Suite 'Selected tests' passed at 2025-01-26 22:33:13.477.
Without GRDBCIPHER
Test Suite 'GRDBPackageTests.xctest' passed at 2025-01-26 22:38:59.398.
	 Executed 23 tests, with 0 failures (0 unexpected) in 314.684 (314.686) seconds

@marcprux the Android NDK doesn't seem to bundle sqlite3.h, whereas GRDB and your PR do rely on that being available. How are you providing that right now when building and testing GRDB?

@marcprux the Android NDK doesn't seem to bundle sqlite3.h, whereas GRDB and your PR do rely on that being available. How are you providing that right now when building and testing GRDB?

It is included with the SQLCipher source module: https://github.com/skiptools/swift-sqlcipher/tree/main/Sources/SQLCipher/sqlite

The results over a dozen runs are a fairly consistent slowdown of around 10% when using SQLCipher. It'd be interesting to tweak the SQLite build flags in swift-sqlcipher to see what might increase/decrease the performance.

Isn't it just the cipher? (EDIT: I'm wrong because performance tests don't run on encrypted databases, see comment below.)

SQLCipher can also be extremely slow when it opens a connection. Some people have reported up to 0.5 seconds (😱). To the point I had to ship #1350.

I ran the same performance tests, but this time in release configuration:

echo "With GRDBCIPHER"; rm -rf .build; GRDB_PERFORMANCE_TESTS=1 GRDBCIPHER="https://github.com/skiptools/swift-sqlcipher.git#1.2.1" swift test --configuration release --filter GRDBPerformanceTests 2>&1 | grep --after-context=2 "Test Suite 'GRDBPackageTests.xctest' passed"
echo "Without GRDBCIPHER"; rm -rf .build; GRDB_PERFORMANCE_TESTS=1 swift test --configuration release --filter GRDBPerformanceTests 2>&1 | grep --after-context=2 "Test Suite 'GRDBPackageTests.xctest' passed"

With GRDBCIPHER
Test Suite 'GRDBPackageTests.xctest' passed at 2025-02-01 13:53:37.700.
	 Executed 23 tests, with 0 failures (0 unexpected) in 72.258 (72.260) seconds
Test Suite 'Selected tests' passed at 2025-02-01 13:53:37.700.
Without GRDBCIPHER
Test Suite 'GRDBPackageTests.xctest' passed at 2025-02-01 13:57:26.264.
	 Executed 23 tests, with 0 failures (0 unexpected) in 80.139 (80.141) seconds
Test Suite 'Selected tests' passed at 2025-02-01 13:57:26.264.

This time, SQLCipher tests are faster.

SQLCipher can also be extremely slow when it opens a connection

Agreed, but the performance hit should only occur when encryption is actually enabled on the database, which shouldn't affect any of the perf tests.

This time, SQLCipher tests are faster.

Of course! Foolish of me to not do that. Otherwise, it is comparing the debug build of the sqlcipher/sqlite source build against the release build of the vendored sqlite.

It is heartening to see that superior performance can be squeezed out of a source build. I wonder how much more might be managed with strategic performance-related flags (e.g., sbooth/CSQLite#59 (comment)).

Apologies for the long delay in addressing the feedback (I was traveling).

I believe I've handled each of the requested changes. Let me know if I can do anything else to help get this PR over the finish line – it has grown quite large, so I'm nervous that rebasing is going to be difficult if it starts to become stale.

We've been using this branch successfully.

I'll follow up on the list of needs at https://mastodon.social/@[email protected]/113990450491505387 here, just to keep it in the PR record.

• Provides built-in SQLCipher support that works for 80+% of people.

This should work out of the box for anyone that just changes:

    dependencies: [
        .package(url: "https://github.com/groue/GRDB.swift.git", from: "7.0.0"),
    ]

to the (theoretical) locally-maintained fork:

    dependencies: [
        .package(url: "https://github.com/groue/GRDB-SQLCipher.swift.git", from: "7.0.0"),
    ]

• No extra dependency (we must control the SQLCipher compilation options).

You are welcome to relocate https://github.com/skiptools/swift-sqlcipher.git into the groue/ org and use that – then you would have control over releases and flags.

• No binaries (they rot).

swift-sqlcipher.git is a pure source build. There are no binaries.

• A clear documentation that tells users what to do to get GRDB+SQLCipher with SPM, written for people who are not build experts.

This isn't part of this PR, but it would only be a couple of lines in the README.

• Testable in GitHub CI.

This is part of the PR. The new SPMSQLCipher job just does:

GRDBCIPHER="https://github.com/skiptools/swift-sqlcipher.git#1.2.1" swift test

• Runs the full GRDB test suite with unencrypted databases, AND with encrypted databases (we already do that for SQLCipher with CocoaPods).

The tests do run against SQLCipher, but against an unencrypted database. I didn't see how this is being done with SQLCipher for CocoaPods, but I suppose we could just do the same setup here (perhaps just based on checking the GRDBCIPHER environment variable)

• Requires minimum efforts for debugging (one can debug tests from Xcode).

This shouldn't be an issue. Just activate (i.e., un-comment) the GRDBCIPHER line in the Package.swift and run the test cases in Xcode.

• Requires minimum efforts for maintenance (i.e. bumping SQLCipher version and adjusting compile options when needed)

For the SQLCipher dependency, you just need to make a new amalgamated sqlite.c from sqlcipher, commit it, and then tag a release. This is done with the script: https://github.com/skiptools/swift-sqlcipher/blob/main/scripts/build_sqlcipher.sh

An example of this sort of update commit is the bump to sqlite 3.46.1 and sqlcipher 4.6.1: skiptools/swift-sqlcipher@5ea2f8b

For GRDB itself, my proposal is that you maintain a fork of GRDB.swift.git at https://github.com/groue/GRDB-SQLCipher.swift.git. The fork is pristine, with the one exception that the GRDBCIPHER line in the Package.swift is un-commented. Whenever you make a release of GRDB.swift.git, you just update the GRDB-SQLCipher.swift.git fork and sync the tags.

It is a little bit of overhead, but it could be added as another step in any scripts you might already be using to maintain releases.

• A technique thats enables SQLCipher support in companion packages as well (GRDBQuery, GRDBSnapshotTesting, RxGRDB).

This is certainly a shortcoming of this method. I unfortunately don't see any solution other than to also have a GRDBQuery-SQLCipher that depends on the GRDB-SQLCipher.swift.git fork.

This last point is indeed painful. However, I think this PR gets us a lot, with minimal intrusion on the existing GRDB structure. A more "correct" solution would be along the lines of #1701, which requires a large refactoring and had performance implications, and so was rejected.

We can have a working GRDB-SQLCipher implementation (plus Android support!) today with this PR, while still continuing to design a more ideal future implementation.